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Abstract: Online continuing education (CE) holds promise as an effective method for rapid dissemination of emerging 
evidence-based practices in health care. Yet, the field of CE continues to develop and delivery is predominately face-to-face 
programs. Practice-oriented online educational methods and e-learning platforms are not fully utilized. Educational 
theorists suggest an experiential approach to CE consistent with adult learning theory. A compelling question remains: Can 
online asynchronous CE programming prepare health care providers in delivering higher-level practice competencies?. To 
address this question, the authors have identified seven composite "sticky" factors that have been critical to the 
engagement of learners and the creation and delivery of practice-oriented online educational programs (Zaghab et a I, 
2015). The sticky factors are based in knowledge management (Nonaka, 1994; Szulanski, 2002) and adult education or 
andragogy (Knowles, 1970; 1984). In this paper, sticky factors are mapped to Moore and colleagues' (2009) higher level 
learning outcomes in health care CE. Data are presented on learner reported practice-related outcomes in a selection of 
online CE courses on the CIPS Knowledge Enterprise™ portal with the University of Maryland School of Pharmacy's Center 
for Innovative Pharmacy Solutions (CIPS). A dynamic, adaptive e-learning environment built by technology partner, 
Connect for Education, Inc., provides the innovative platform and the Acclaim! interactive learning technology. This 
technology-instructional partnership is dedicated to an iterative continuous improvement process called the Learner 
Stewardship Cycle (Zaghab et a I, 2015). The cycle improves stickiness and learner engagement in order to achieve higher- 
level learning outcomes in CE. Findings suggest that of the 769 learners successfully completing an online course with two 
or more sticky design segments, the majority report reaching level 4, 5 and 6 learning competencies. Learners from the 
professions of pharmacy, nursing, medicine, and other health decision makers also found the courses relevant, easy to use 
and evidence-based. 

Keywords: Health Care Practitioner, continuing education, situated online learning, learner engagement, continuous 
improvement, and practitioner-learner 

1 Background - Challenges in online continuing education 

While emerging options in the online delivery of education have expanded exponentially in the last decade, 
health care continuing education (CE) offerings have not kept pace with the capabilities offered (Furman & 
Sibthorp, 2013). Health care is a rapidly changing field driven by the extraordinary pace of scientific discovery 
and CE is an important way that health professionals are "updated" with the latest knowledge and skills. New 
scientific discoveries often require changes in clinical practices. However, practice is slow to change (Ferlie et 
al, 2005) and CE professionals are stepping into this gap. Pressure is mounting to expand the use of dynamic 
online platforms for CE to meet health professional needs (ACEHp, 2015). 

Online asynchronous programs allow for standardized, geographically dispersed learner enrollment anytime 
and anywhere without cost of travel. When compared to live programming, online CE is an accessible, 
convenient, and often cost-effective alternative. Asynchronous CE is also a viable option to disseminate new 
evidence and best practices to a broad audience of health professionals. 

A global meta-analysis conducted by Cook and colleagues (2009) compared the effectiveness of 201 online CE 
programs to live programs to find no significant difference in the effectiveness of online versus face-to-face CE 
in achieving the stated learning outcomes. This meta-analysis compared learner outcomes in internet and 
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face-to-face delivered CE programs over a 17 year period of time. However, the study did underscore the 
variation in the quality of the CE programming. 

Even in agencies charged with practice dissemination, e-learning technologies are not fully utilized. One 
example of this is the CE programs offered by the Effective Healthcare Program of the U.S. Agency for 
Healthcare Research Quality, the agency responsible for dissemination of evidence-based practice change 
( http://effectivehealthcare.ahrq.gov/tools-and-resources/cmece-activities/ ). The Effective Healthcare Program 
offers a total of 41 continuing education modules targeting physicians (41), pharmacists (14), nurses (14), 
nurse practitioners (14), case managers (14) and health educators (6). The educational strategies for AHRQ's 
CE offerings included readings from medical publications, visual (not audio) slide presentations, and multiple 
choice post-tests. Three of the 41 modules were playback webinars (7.3%) and eight (19.5%) programs had 
playback audiovisual. None of the 41 modules on dissemination of best practices in effective health care were 
interactive. Six workshops were delivered face-to-face in locations across the U.S. 

2 Measuring health practitioners' needs as learners 

The learning orientation of health care practitioners presents particular challenges for asynchronous online CE 
programming. Practice, by definition, is a series of interconnected behaviors including: "forms of bodily 
activities, forms of mental activities, "things" [tools and artifacts] in their use, a background knowledge in the 
form of understanding, know-how, states of emotion and motivational knowledge." (Reckwitz, 2002, p. 249). 
In short, practitioners require practice in order to learn to practice. 

Moore, Green and Gallis (2009) built and have improved a widely accepted framework for the measurement of 
CE program success in achieving practice-oriented outcomes for health professionals. In their framework, 
learning for practice is connected to "doing". Thus, effective CE programs should transmit actionable 
knowledge and provide a forum for learners to demonstrate physical competencies. Table 1 summarizes the 
outcomes to be measured in health care CE programming. These outcomes serve as metrics for the CE activity 
evaluations reviewed in this paper. 

When designing online programs, quality factors for experiential CE point to the learning outcomes framework 
proposed by Moore, Green, and Gallis (2009). The learning outcomes for medical practitioners, argued Moore, 
must extend beyond declarative or domain knowledge to include process-type knowledge such as adhering to 
clinical practice guidelines when treating patients. Ultimately, practitioners must demonstrate competency 
(level 4) and the ability to perform patient care in an authentic setting (level 5). In addition to command over 
domain knowledge. Levels 5 and 6 CE learning outcomes involve mobilization of embodied knowledge. Learner 
simulation of practice skill in a trial-and-error setting is important for practitioner-learners to ultimately 
achieve these learning outcomes online. 

Health care is not the only field where learning is experiential. Adult learning theorists endorse educational 
methods with relevant, actionable knowledge for all adult learners (Knowles, 1984). Adults who are rich in 
practice experience must have opportunities to demonstrate the application of learning to authentic work 
settings. 

As noted in an earlier publication (Zaghab et a I, 2015), many learning management systems and online 
learning environments (Hassanzadeh et al, 2012; Wang, Want & Shee, 2007; Liu, 2014) fail to measure the 
engagement of the learner, and the inclusion of situated, contextual learning, active learning components, and 
authentic learning environments. Table 3 identifies program design factors from the literature and shares 
implications for online engaged learning. Online CE can be designed with higher level learning competencies as 
the desired outcomes. Design factors can be, and in the study today, are, integrated into program design to 
meet the CE needs of adult practitioners. 
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Source 

Educational factors identified 

Implications for course 
design 

Martz & 

Shepherd 

(2003) 

Demonstration of skill is required. Course design must address not only 
changes in content or domain knowledge, it must address behaviors 
associated with practice change. 

Active learning exercises 
encourage practice. Hands- 
on practice demonstrates 
competency. 

Herrington & 
Oliver (2000) 

Authentic learning environments are essential for practitioners. Learning 
and the context of learning cannot be separated. Knowledge must be 
"relevant to solve problems"- not to memorize facts. Praxis creates 
actionable knowledge for practice. 

Learning environments 
simulate authentic settings 
for professional practice and 
provide evidence-based 
tools for clinical decision¬ 
making. 

Brown, Collins 
& Duguid 
(1989) 

Practitioners learn in a way that will prepare them for complexity and 
performance in real life. 

Complex case studies and 
real-life challenges are 
opportunities for critical 
thinking and translation of 
knowledge into practice. 
Feedback on case work is 

formative. 

Moore, Green 
& Gallis 
(2009) 

Techniques to reinforce learning include: a) creating and reinforcing a 
learning experience as a "teachable moment"; b) enabling practitioner- 
learners to utilize critical thinking and reasoning in treatment decisions; 
c) structuring an experience to elicit learner's commitment to improving 
competency; and d) providing formative feedback to learners. 

Utilize experts to testify, 
provide relevance, and 
model behavior prior to 
learner demonstration. 

Osman & 

Hannafin 

(1992) 

Metacognition or self-assessment by learners is an ongoing process to 
improve practice. Learners interpret and generate inferences. Reflection 
facilitates meta-learning. 

Reflection exercises relate 
to the performance or 
practice exercises by 
learners or peers. 


Table 1: Literature on experiential education: Implications for educational program design 
excerpted from (Zaghab et a I, 2015) 


3 Structure and purpose of the paper 

This paper will review the factors in experiential learning for health care professionals. Next the authors will 
apply these factors to online learning environment in light of Moore, Green and Galli's long-standing 
framework for CE competencies. Sticky aspects of e-learning environment will be described with regard for 
how to address the unique challenges of designing and delivering practice-oriented online CE for health 
professionals. The setting for the study will be described. Program evaluation data from 769 CE learners and 
87 sticky course learners will report descriptive learning outcomes according to Moore's framework. The 
learners' perceived outcomes and competencies are noted. Findings will be discussed and limitations will be 
noted. 

4 Research question and definitions 

The question to be addressed in this research is: When e-learning design factors are optimized in evidence- 
based online CE programs, can health care practitioners consistently achieve high levels of learning outcomes? 

Knowledge is extremely useful in informing clinical practice, the CIPS Knowledge Enterprise philosophy aligns 
with Moore and company regarding the importance of an evidence-based transfer of knowledge and also the 
importance of integrating such evidence into embodied knowledge for clinical action demonstrated in Moore's 
levels 4, 5 and 6. 


www.ejel.org 


468 


©ACPIL 











Roxanne Ward Zaghab, Carlos Maldonado, Dongsook Whitehead et al 




Proposed sticky factor 

Online manifestation of the factor 



A. Active learning 

Active learning exercises encourage practice. Hands-on 



exercises focus 

practice demonstrates competency. Authentic settings are 



on competencies 

key to the transfer of tacit knowledge into practice skills. 

E 


B. Multimedia 


V 


demonstrations of 

Experts provide testimonial and can be engaged in 

E 

CD 

U 

c 

skills 

modeling or demonstrating processes. 


CD 



4 

CD 

Q. 

£ 

C. Role reflection 

Explore care roles from a variety of perspectives. 


u 

D. Self-assessment 

Solve complex problems individually and with a team. 

L 

CD 

E. Real-world tasks 

Activities are true to the tasks and experience of the 

E 

o 

’+-> 

in authentic settings 

practitioner. 

E 

CD 

i_ 

CL 

F. Practice coaching 

Non-linear instruction represents the complexities of real- 

L 

C 


world practice. 

5 

1^2 

CO 

G. Authentic practice 

assessments 

Authentic assessments of knowledge, skills and abilities. 


Table 2: "Sticky" online program design and delivery based on Moore's CE outcome levels (Adapted from 
Zaghab et al, 2015) 

Moore and colleagues' framework on the levels of continuing education outcomes has been widely accepted 
and serves as the basis for our approach. The framework was developed primarily for medical education but 
has been applied broadly to the health professionals (AChEP, 2015). Moore's framework includes: 

• Level 1 tallies nothing more than participation or attendance. 

• Level 2 notes learner satisfaction. 

• Level 3 focuses on the learner expectations and the level to which the course meets or exceeds those 
expectations. Level 3 includes two types of knowledge: a) declarative knowledge or ability to recall 
facts; and b) the learners' ability to restate how to do what the CE activity intended. 

• Level 4 examines competencies gained by the learner through the CE experience, specifically how to 
demonstrate in an educational setting the knowledge transmitted in the CE activity. 

• Level 5 targets the learner's ability to translate the knowledge or skill into the performance of 
practice. 

• Level 6 reaches the patient experience and seeks benefit to the patient's condition as a result of the 
health practitioner's action. 

• Level 7 is the degree to which the health status of the community changes. 

The focus of this paper is on Levels 4 (Competence), 5 (Performance) and 6 (Patient Health Outcomes). 

The paper links sticky factors in module design in the e-learning environment with the learner reported 
achievement of these levels. 

The term "sticky" utilized in Table 2 is borrowed from Szulanski (2000), who coined the term and articulated 
the challenges of transferring tacit knowledge between individuals in the workplace. His research 
substantiates that "knowing in practice" is the most difficult kind of knowledge to transfer. In this paper, tacit 
knowledge is known as embodied knowledge or knowledge that is held in experience. Examples include: how 
to give an injection, conduct an ultrasound, and perform a surgical procedure. 

The term "sticky" is borrowed from Szulanski. Szulanski's (1994) research found that tacit knowledge "sticks" 
to the owner and rejects transfer unless it is accompanied by embodied experiences. Since early research in 
knowledge transfer, innovative educators have shed light on situated learning and practice-based learning as 
noted in Table 2. The literature delineates desirable instructional factors in situated learning (Martz & 
Shepherd 2003; Moore et al 2009). This work extends and organizes these factors into educational elements 
necessary for design, delivery and evaluation of experiential learning in online learning environments. 
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5 Measures and methodology 

This real-world investigation examines learner-reported learning outcomes in a total of 769 CE activity 
evaluations submitted by learners enrolled in the CIPS Knowledge Enterprise portal between November 2012 
and October 2015. Electronic surveys via Survey Monkey were administered to learners immediately following 
successful completion of each course. Course instructors were blinded as to the learner's completion and 
responses. Completion rate of the survey was 100% for health professionals seeking CE credit because the 
activity evaluation was required to receive CE credit. Courses were certified for pharmacist, pharmacy 
technician, physician, and school nurse participation, and more recently physicians. Two hybrid courses had a 
final course component by phone or face-to-face. 

The activity evaluations maintained consistent elements for reasons of certification but varied according to the 
learning outcomes of the modules. Two forms of Likert Scale questions (four point and five point scales) were 
used and these variations are noted in the analysis. Four point Likert questions were adjusted with zero for 
neutral in calculating a mean score. Only one survey response was allowed per learner and no identifiable 
learner information was used. Thus, learners may have participated in one or more courses, a matter not 
considered in this analysis. It may have influenced the learner's cumulative competencies- an area for future 
research. 

The course mix totaled 87 CE hours involving 24 courses. Of these courses 19 included active learning 
exercises, 20 involved multimedia, 6 practiced active self-assessment, 7 provided real-world tasks and cases, 
14 required skills practice, 12 guided learners in role reflection, 4 involved practice coaching, and 15 involved 
real-world assessments. 

Online courses selected for this analysis include two or more segments with sticky design factors CE activities 
ranging from 1-20.5 hours of credit. Courses, series of courses, and practice certificate programs were selected 
based on the factors in Table 2. Qualifying design factors included practice activities, interactive problem- 
based learning, hands-on skills practice, practice skill demonstrations with stakeholders, either assessments or 
practice using real-world cases, self-assessment through reflection, and application of actionable knowledge 
such as clinical guidelines to complex clinical examples. In certificate programs and a handful of skill-based 
online courses, e-learning technologies have enabled individualized skill demonstration and individualized 
coaching. 

Each CIPS Knowledge Enterprise course (module) is comprised of many segments combined to achieve the 
application and practice-based learning objectives. Components include but are not limited to the design 
factors in Table 3: a) instructional materials; b) faculty introductions, disclosures; c) learning objectives and 
description of the larger series or certificate; d) testimonials from experts in the field as well as practitioners 
on the application; e) multimedia knowledge-based instruction (also referred to as micro-lectures); f) correct 
and incorrect video demonstrations of practice activity; g) active learning complete with interactive question 
and answers as well as discussion; g) professional self-reflection; h) hands on demonstrations of learners for 
skills practice; i) complex case studies including patient videos, medical charts for practitioner intervention; 
and j) case studies requiring the application of clinical guidance. 

Each online course consists of artful rather than prescribed combination of possible segments. Modules vary in 
how sticky segments are designed and combined to achieve the learning outcomes. Only courses with two or 
more segments with sticky designs were included in the study. 

6 The theoretical perspective 

The authors adopt a social constructivist epistemology of teaching and learning (Palincsar, 1998) in this 
examination of learning outcomes and course design. The sticky design and related technologies when 
combined act as the mediator between learner and the e-learning environment. This is similar to the dominant 
role the instructor plays in a face-to-face classroom. Thus, sticky design is considered by the authors, to be a 
compelling theoretical factor in how e-learning can achieve practice-related competencies in online learners. 
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7 The setting 

Since 2011, the University of Maryland School of Pharmacy's CIPS and Connect for Education, Inc. (C4E) have 
collaborated in a university-technology partnership. The CIPS Knowledge Enterprise™ 

(www.pharmacists4knowledge.org) is a mobile-accessible, asynchronous, continuing education portal for 
practicing health professionals. The CIPS Knowledge Enterprise uses a quality improvement approach termed 
the Learning Stewardship Cycle (LSC) (Zaghab et a I, 2015), a step-by-step process of micro-improvement to 
improve the stickiness of the online program based on data provided by the e-learning environment. The LSC 
micro-improvements aim to optimize stickiness and learner engagement for the practitioner-learner in the 
online asynchronous classroom. 

The portal offers e-commerce services including registration, payment, and certification history for learners. 
Since its inception the portal has offered over 550 continuing education hours, many practice series, and four 
practice-based certificate programs with additional practice certificates planned. Enrollments have been 
concentrated regionally, but numbers include learners from across the globe, including physicians, 

pharmacists, school nurses, pharmacy students, pharmacy technicians, and health decision-makers. Working 
side-by-side with content experts, the CIPS-C4E team plans and shapes segments carefully to ensure that 
content is optimized through the use of sticky factors in design. Higher level learning outcomes require more 
rigorous educational plans. One powerful capability offered in the e-learning environment is Acclaim™, a 
virtual simulator. The CIPS Knowledge Enterprise provides individualized coaching in the form of formative 
feedback via Acclaim! 

CIPS Knowledge Enterprise online CE courses are created from building blocks or "segments" which range in 
length from 15 seconds to 15 minutes. Segments include: expert testimonials, program mapping, audiovisual 
didactic presentations, interactive inquiries, self-reflection, learner feedback on exercises, regular learner 
reinforcements, active learning exercises, audiovisual case studies, skill demonstrations, practice skill 
exercises, scaffold case video scenarios, mailed inhaler placebo devices, videotape help tools, instructor 
coaching and feedback session, hands-on video simulation (assessment), and knowledge-based assessment, 
among others. Educational programs range from one to over 20 hours. As the length of the program grows, so 
does the importance of improved stickiness in design. 

8 Findings 

Drawing from a sample of 769 learners in sticky designed courses, the findings in this section suggest patterns 
and emerging trends in the relationship between sticky design and learner reported level 4, 5 and 6 outcomes. 
This section will provide a summary of CE offerings in the form of practice certificates (15 or more hours), 
series (multiple modules focused on application of skills), and course offerings with two or more sticky factors 
integrated into the design. Given the importance of academic integrity, learner reports of the evidence basis 
and credibility of the course is provided. Differences among health care professionals are noted. Next the 
learner reported competencies and outcomes will be presented. 

Balance, objectivity, and a current base of scientific evidence are all foundational elements in any certified 
continuing educational course for health professionals. Table 3 presents learner perceptions of course 
credibility, accessibility, and overall satisfaction. The mean responses are aggregated for both 4 and 5 point 
Likert questions for all sticky courses. 

Table 3: Learner Perceptions of Module Integrity and Academic Credibility 


Statement 

N 

Mean 

N 

Mean 

Easy to use, accessible, convenient 

514 

4.3 

93 

3.7 

Training was evidence-based 

590 

4.4 



Balanced and objective 

676 

4.5 

18 

3.8 

Satisfaction with module 

590 

4.3 
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The mean score for credibility was 4.4; for objectivity a mean of 4.5; and a mean of 3.8 was reported. Learners 
indicated agreement and strong agreement for ease of use (4.3 and 3.7) and overall satisfaction with the 
online CE course (4.3). As noted in the methods, the post-course surveys varied slightly which accounts for the 
differences in numbers. 

Physician satisfaction, in general, ranked the e-learning environment ranked lower (mean 3.6) than 
nurses/pharmacists (mean 4.5) and pharmacists alone (mean 4.4). Other health professionals and decision 
makers ranked the e-learning environment as 3.9 which indicated a slant toward agreement but some level of 
neutrality. 

Table 4: Appropriateness of e-learning environment as ranked by profession 


Professional Affiliation 

Pharmacists 

Nurses and pharmacists 

Physicians 

Healthcare Decision-Makers 
Question Not Asked 
TOTAL 


N 

Mean 1 

Mean" 

446 

4.5 


144 

4.5 


18 


3.6 

75 

3.9 


86 



769 

4.3 

3.6 


Learner reported benefits from the sticky courses are broken down by number and percent of respondents in 
each Likert measure. Level 6 learning outcomes were not achieved by every learner. Ninety-seven point seven 
of 679 respondents reported the competencies and skills from the course will impact their patients positively. 
Only 2.7% disagreed. A similar impact was reported by 635 learners where 96.8% agreed that the course 
improved their clinical service delivery. Only 3.3% reported disagreement or strong disagreement with a 
perceived impact on their clinical practice. While 1.3% of the learners disagreed and did not feel more 
confident in applying learning to problem decisions in their authentic work environment, 94.7% reported 
agreed or strongly agreed that they felt more confident after the online training. 

A similar pattern is found in level 5 learning outcomes where 92% of 567 learners agreed or strongly agreed 
that the case studies had practice-relevant application. This question is unique in that it directly associates a 
single sticky course design element to a specific learning outcome. Level 4 learning outcomes are also essential 
to the learner-practitioner's experience. In a cohort of 75 hybrid course learners, 93.3% found the course 
design and concepts applied to real decisions. 

Learners were asked to rate the appropriateness of the online environment for practice-based CE. There were 
notable differences between professions. The physician ratings ranked between agree and neutral (3.58) with 
only 15 learners reporting. Decision makers (mean 3.93) included physicians, consultants, nurses, and non¬ 
pharmacists. Whereas, pharmacists and mixed cohorts of nurses and pharmacists ranked higher with a mean 
of 4.36 and 4.57, respectively. 

In a longitudinal study of an interprofessional cohort of health decision-makers, Pickering and colleagues 
(Pickering et al, 2015) found that learners in the practice certificate rated real-world decision making as 4.04 
mean, in the hybrid program in comparative effectiveness research. Learners felt confident of their abilities to 
impact real work decision-making and to solve problems in their authentic work setting- both indicators of 
Moore's Levels 5 and 6 competencies. 
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Table 5: Sticky courses and learner reported ability to achieve Moore's Level 4, 5 and 6 CE learning outcomes 



Total 

N 

Learner Self-Reported Benefit 

Strongly 

Disagree 

Disagree 

No 

Opinion 

Agree 

Strongly Agree 

N 

% 

N 

% 

N 

% 

N 

% 

N 

% 

My patients will 
benefit from my 
new knowledge 

gained from the 
training. (Level 6) 

676 

5 

0.7% 

9 

1.3% 

0 

0% 

343 

50.7% 

319 

47.2% 

The case studies and 

interactive exercises 
helped me to apply 
the information 

provided. (Level 5) 

567 

7 

1.2% 

36 

6.4% 

2 

0.4% 

303 

53.4% 

219 

38.6% 

The program will 
help me better 
deliver high quality 
clinical services. 

(Level 6) 

635 

3 

0.5% 

18 

2.8% 

0 

0% 

307 

48.4% 

307 

48.4% 

The course design 
and materials 

showed how course 
concepts could be 
applied to real 
decision problems. 
(Level 4) 

75 

0 

0% 

5 

6.7% 

4 

5.3% 

49 

65.3% 

17 

22.7% 

1 feel more confident 
in my ability to 
provide valuable 

input...into problem 
decisions in my work 
setting. (Level 6) 

75 

0 

0% 

1 

1.3% 

3 

4.0% 

48 

64.0% 

23 

30.7% 


9 Conclusion and Study Limitations 


This paper is the first step in exploring the complex interrelationship of higher-level learning outcomes in CE 
and sticky course design. Research into sticky factors can become more robust in a number of ways. 
Benchmarking learner progress is a challenging task, especially for level 5 and 6 outcomes. In the future, in 
order to adequately determine how the CE course has changed practice, we must first assess the status of the 
learner as well as a sense of their practice environment. A pre-course self-report of practice setting could be 
considered. 

To provide needed power for statistical analysis, standardization of the five point Likert scale in activity 
evaluations is necessary and already underway. The differences between health professionals can be better 
assessed within the same course, series, or certificate and is another area for future research. With a larger 
sample size, each specific sticky factor could be analyzed separately to determine its effectiveness in achieving 
level 4, 5 and 6 learning outcomes. From the continuous improvements and micro-changes discussed in the 
Learning Stewardship Cycle, CIPS Knowledge Enterprise courses create a unique set of challenges. Continuous 
improvement suggests that the course at the beginning may not have been as "sticky" as the same course 
evaluated at the end of the period under study. 

Cognizant of the limitations, the authors conclude that the vast majority of the CE learners reported practice¬ 
relevant outcomes which will benefit their patients, their practice or their decision-making abilities. This 
meaningful contribution may open doors for sticky e-learning as a method for widespread dissemination of 
practice-relevant CE for health professionals. Public, private and governmental units dedicated to 
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dissemination of best practices should critically assess the design and delivery of their CE programs with the 
sticky factors in mind. 

Sticky factors in e-learning course design are concepts in need of additional definition. The authors urge the 
development of new metrics and the validation of metrics for each sticky factor in online course design. Once 
validated, new metrics could serve as a source of guidance for any CE provider who sets sights on optimizing 
the stickiness of their CE programs. 
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